40 research outputs found
A Bregman forward-backward linesearch algorithm for nonconvex composite optimization: superlinear convergence to nonisolated local minima
We introduce Bella, a locally superlinearly convergent Bregman forward
backward splitting method for minimizing the sum of two nonconvex functions,
one of which satisfying a relative smoothness condition and the other one
possibly nonsmooth. A key tool of our methodology is the Bregman
forward-backward envelope (BFBE), an exact and continuous penalty function with
favorable first- and second-order properties, and enjoying a nonlinear error
bound when the objective function satisfies a Lojasiewicz-type property. The
proposed algorithm is of linesearch type over the BFBE along candidate update
directions, and converges subsequentially to stationary points, globally under
a KL condition, and owing to the given nonlinear error bound can attain
superlinear convergence rates even when the limit point is a nonisolated
minimum, provided the directions are suitably selected
A Simple and Efficient Algorithm for Nonlinear Model Predictive Control
We present PANOC, a new algorithm for solving optimal control problems
arising in nonlinear model predictive control (NMPC). A usual approach to this
type of problems is sequential quadratic programming (SQP), which requires the
solution of a quadratic program at every iteration and, consequently, inner
iterative procedures. As a result, when the problem is ill-conditioned or the
prediction horizon is large, each outer iteration becomes computationally very
expensive. We propose a line-search algorithm that combines forward-backward
iterations (FB) and Newton-type steps over the recently introduced
forward-backward envelope (FBE), a continuous, real-valued, exact merit
function for the original problem. The curvature information of Newton-type
methods enables asymptotic superlinear rates under mild assumptions at the
limit point, and the proposed algorithm is based on very simple operations:
access to first-order information of the cost and dynamics and low-cost direct
linear algebra. No inner iterative procedure nor Hessian evaluation is
required, making our approach computationally simpler than SQP methods. The
low-memory requirements and simple implementation make our method particularly
suited for embedded NMPC applications
A new envelope function for nonsmooth DC optimization
Difference-of-convex (DC) optimization problems are shown to be equivalent to
the minimization of a Lipschitz-differentiable "envelope". A gradient method on
this surrogate function yields a novel (sub)gradient-free proximal algorithm
which is inherently parallelizable and can handle fully nonsmooth formulations.
Newton-type methods such as L-BFGS are directly applicable with a classical
linesearch. Our analysis reveals a deep kinship between the novel DC envelope
and the forward-backward envelope, the former being a smooth and
convexity-preserving nonlinear reparametrization of the latter
Newton-type Alternating Minimization Algorithm for Convex Optimization
We propose NAMA (Newton-type Alternating Minimization Algorithm) for solving
structured nonsmooth convex optimization problems where the sum of two
functions is to be minimized, one being strongly convex and the other composed
with a linear mapping. The proposed algorithm is a line-search method over a
continuous, real-valued, exact penalty function for the corresponding dual
problem, which is computed by evaluating the augmented Lagrangian at the primal
points obtained by alternating minimizations. As a consequence, NAMA relies on
exactly the same computations as the classical alternating minimization
algorithm (AMA), also known as the dual proximal gradient method. Under
standard assumptions the proposed algorithm possesses strong convergence
properties, while under mild additional assumptions the asymptotic convergence
is superlinear, provided that the search directions are chosen according to
quasi-Newton formulas. Due to its simplicity, the proposed method is well
suited for embedded applications and large-scale problems. Experiments show
that using limited-memory directions in NAMA greatly improves the convergence
speed over AMA and its accelerated variant
QPALM: A Newton-type Proximal Augmented Lagrangian Method for Quadratic Programs
We present a proximal augmented Lagrangian based solver for general convex
quadratic programs (QPs), relying on semismooth Newton iterations with exact
line search to solve the inner subproblems. The exact line search reduces in
this case to finding the zero of a one-dimensional monotone, piecewise affine
function and can be carried out very efficiently. Our algorithm requires the
solution of a linear system at every iteration, but as the matrix to be
factorized depends on the active constraints, efficient sparse factorization
updates can be employed like in active-set methods. Both primal and dual
residuals can be enforced down to strict tolerances and otherwise infeasibility
can be detected from intermediate iterates. A C implementation of the proposed
algorithm is tested and benchmarked against other state-of-the-art QP solvers
for a large variety of problem data and shown to compare favorably against
these solvers
Bregman Finito/MISO for nonconvex regularized finite sum minimization without Lipschitz gradient continuity
We introduce two algorithms for nonconvex regularized finite sum
minimization, where typical Lipschitz differentiability assumptions are relaxed
to the notion of relative smoothness. The first one is a Bregman extension of
Finito/MISO, studied for fully nonconvex problems when the sampling is random,
or under convexity of the nonsmooth term when it is essentially cyclic. The
second algorithm is a low-memory variant, in the spirit of SVRG and SARAH, that
also allows for fully nonconvex formulations. Our analysis is made remarkably
simple by employing a Bregman Moreau envelope as Lyapunov function. In the
randomized case, linear convergence is established when the cost function is
strongly convex, yet with no convexity requirements on the individual functions
in the sum. For the essentially cyclic and low-memory variants, global and
linear convergence results are established when the cost function satisfies the
Kurdyka-\L ojasiewicz property